A Brief Survey on Semi-supervised Learning with Graph Regularization

In this survey, we go over a few historical literatures on semi-supervised learning problems which apply graph regularization on both labled and unlabeled data to improve classification performance. These semi-supervised methods usually construct a nearest neighbour graph on instance space under certain measure function, and then work under the smoothness assumption that class labels of samples change quite slowly for connected instances on the graph. Graph Lapalcian of the graph has been used in the graph smoothness regularization since it is a discrete analogous to Beltrami Laplace smoothness operator in the continuous space. A relationship between kernel method and graph Laplacian has also been discovered, and suggests an approach to design specific kernel for classification utilizing the spectrum of Laplacian matrix. 1 Smoothness Regularization via Laplace Operator and Graph Laplacian 1.1 Semi-supervised Learning Problem Nowadays there are many practical applications of machine learning problem. Though in academic papers we often assume these problems have some nice conditions and huge number of samples for inference and learning, we cannot achieve such datasets in reality. One of the biggest problem is that not all of sample instances have class labels since people can hardly label millions of instances manually. Then we need some learning method that can deal with the problem with both labeld and unlabled data at the same time, utilizing the labels of labeled instances and also the information from unlabeled instances. Let X be the instance space, {x1, x2, ..., xk} are the k labeled training samples with labels {y1, y2, ..., yk}, yi ∈ [−1,+1], and {xk+1, xk+2, ..., xn} are the n−k unlabeled samples. Usually we have k n which means only very few samples are labeled. Our purpose is learning a classification function f : x → [−1,+1] to predict the class of testing samples. We restrict ourselves to the transductive setting where the unlabelled data also serve as the test data in our performance analysis. 1.2 Manifold Assumption and Laplace Operator The information from unlabeled instance is hard to be incorporated into the classification model unless we could raise a reasonable assumption of the unlabeld data. The first assumption is thatthe data lies on a low-dimensional manifold within a high-dimensional representation space. We can easily find many examples to justify this assumption. For example a handwriting digital number can be represented as a matrix of image pixel with very high dimension, however it can also be represented by only a few parameters in the low-dimensional ambient space, and in this space, the structure of similar handwriting digital numbers lie close to each other to form a low-dimensional manifold. Similar examples can be find in document categorization problems, where people discov-